Gradient Cross Correlation for Sub-pixel Matching

Sub-pixel matching is one of the key components for image registration and image fusion. Ideally, image matching should allow for offsets in the target image, and for scaling and rotation. Offsets allow for sub-pixel shifts in the two images, while scaling is necessary when matching images from different sensors or images taken from different distances using the same camera. Rotation allows for the matching between rectified and un-rectified images or images taken from different viewpoints. This paper presents a novel matching method called gradient cross correlation, which has been derived from the well-known normalised cross correlation coefficient formulation. In experimental evaluations, the method has been applied to image matching from various satellites (Landsat MSS and TM, QuickBird and other sensors). For comparison, an alternative method for estimating sub-pixel shifts and scaling and orientation parameters was applied – the least squares matching method. The mathematical details of the gradient cross correlation method, the experimental results, and some aspects of how to implement the approach in practice will be described and discussed in this paper. Various models of the gradient cross correlation have been derived from the relationships between the affine transformation parameters. A hierarchy of relationships between the affine transformation parameters can be specified in practice as follows: Model-I: different scale, different rotation; IIA: different scale, common rotation; IIB: common scale, different rotation; III: common scale, common rotation; and IV: fixed scale, fixed rotation. These models lead to a more natural interpretation of the resulting parameters, especially when matching images which have considerable scaling and rotation differences. The particular formulation of the affine transformation adopted leads to useful insights into the image matching. The experiments showed that Model-IV is the worst model for matching all kind of points. It is essential to choose an appropriate geometric transformation depending on different image characteristics and types of points. The gradient cross correlation method and the least squares matching method with an offset and gain are equivalent from a theoretical point of view. Both methods can achieve sub-pixel matching accuracy and, when appropriate models are chosen, they give very similar results. However, from an implementation point of view, the gradient cross correlation method is superior to the least squares matching method because radiometric correction and geometric correction can be achieved using only scaling and rotation parameters. Furthermore, incorporating a line search strategy with either the gradient cross correlation method or the least squares matching method shows that improved cross correlation coefficients may be achieved within a few extra iterations. Experiments were conducted to compare methods applied to a range of images. The matching correlation results from the gradient cross correlation are nearly identical (both the matching results and the number of iterations) to that of the least squares matching. However, the gradient cross correlation method combines radiometric correction and geometric correction into a single step, which makes its parameter estimation and practical computation implementation simple. Both the gradient cross correlation method and the least squares matching method require good initial approximations or a small pull-in range in order to find the minimisation points (1 to 2 pixels in average from our experience). For the matching of raw to rectified TM images, the scaling is about 0.83 (25m/30m) and is the same for line and pixel, while the angle of rotation is common for line and pixel, at around 10°. For the matching of raw MSS to rectified TM images, the angle of rotation is common for line and pixel, again at around 10°, while the scaling is different for line and pixel, agreeing closely with the expected values of 0.44 (25m/57m) and 0.32 (25m/79m), respectively. Reasonably good results were also obtained when points were matched from QuickBird to SPOT and to TM, given the huge pixel resolution change (more than 40 times between QuickBird and TM). For matching of a stereo pair of high-resolution images, the flexibility of varying the scaling and/or orientation gives a better matching correlation. It could be valuable to use bootstrap procedures to establish the typical range of variation for the matching correlation for Model-I against which to judge the adequacy of the simpler models.